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A Tier 1 Intervention to Increase 9‘"-Grade Engagement and Success: Results from a 
Randomized Controlled Trial 
Abstract 

Although high school graduation rates are improving, a large number of students are still not 
successful. Research has documented that 9th grade is a pivotal year in determining whether a 
student will graduate or drop out. The purpose of this randomized controlled trial was to assess 
the effects of a Tier 1 intervention model (Freshmen Success) for 9" graders to increase school 
engagement, attendance, credits earned, and grade point average (GPA). This study included 
1,588 students in 9th grade across four comprehensive high schools. Treatment schools 
implemented the Freshmen Success components: a 9"" grade Leadership Team, a curriculum, and 
support from Peer Navigators. Control schools continued business as usual. Results showed 
statistically significant and educationally meaningful effects on student motivation, engagement 
and attendance, and a moderate-to-large effect for credits earned; however, there was no 
significant effect found for GPA. 
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Impact and Implications Statement 

This study demonstrates that the FS model, when implemented with fidelity, increased 9™ 
grade student engagement, attendance, and credits earned. However, the intervention was not 


associated with significant improvements in GPA. 


A Tier 1 Intervention to Increase 9‘"-Grade Engagement and Success: Results from a 
Randomized Controlled Trial 

Although graduation rates are improving nationally, too many students still do not 
complete high school. School dropouts, when compared with their graduating peers, are more 
likely to be unemployed or underemployed, live in poverty, have poor health, and become 
involved in criminal activities (Belfield, Levin, & Rosen, 2012; Christle, Jolivette, & Nelson, 
2007; McFarland, Cui, & Stark, 2018). Research has documented that the path to dropping out is 
a gradual process of diminishing school engagement (Reschly & Christensen, 2012) and, for 
many, begins with the transition from middle to high school (Allensworth & Easton, 2005; 
Benner, 2011; Somers & Garcia, 2016). In particular, 9'"-graders have been shown to have lower 
attendance rates (Jerald, 2006), higher rates of disciplinary action (Flannery, Fenning, McGrath 
Kato, & Bohanon, 2013; Kaufman et al., 2010; Spaulding et al., 2010) and lower course 
performance (Allensworth & Easton, 2007; Roderick, Kelly, Kemple, Johnson, & Beecham, 
2014) than their older peers. In fact, Allensworth and Easton (2007) found that students who fell 
behind in 9" grade had a graduation rate 60% lower than that of students who were able to stay 
on track during the 9""-grade year. 

To prevent high school dropout, a growing body of research supports the implementation 
of multi-tiered systems of support (MTSS) to foster student engagement, positive social 
interactions, and academic achievement for all students (Brown-Chidsey & Steege, 2010; Goss 
& Andren, 2014). MTSS provides a framework for schools to implement evidence-based 
interventions as they supply (a) systems needed for initial and sustained implementation, (b) 
guidance in the selection and implementation of practices that match the needs of the school, and 


(c) systems for using data to identify areas of concern and guide decision-making regarding 


interventions (Ervin, Schaughency, Matthews, Goodman, & McGlinchey, 2007; Sugai & Horner, 
2009). This continuum of supports begins with the whole school and becomes more intensive 
and individualized based on student need. Tier | (i.e., universal, schoolwide) emphasizes 
prevention and is designed for all students, adults, and school contexts. The Tier 2 (i.e., targeted) 
consists of efficient interventions offered to groups of students who need additional support. 
Finally, Tier 3 (1.e., individualized) provides the highest need students with intensive supports 
(see www.pbis.org). These preventive models have been shown to be successful in high schools 
by reducing problem behavior, increasing attendance, and improving student access to needed 
interventions (Bohanon et al., 2006; Flannery, Fenning, McGrath Kato, & McIntosh, 2014; 
Morrissey, Bohanon, & Fenning, 2010; Muscott, Mann, & LeBrun, 2008) and also have recent 
application to dropout prevention and transition programming (Dynarski et al., 2008; Furlong & 
Christenson, 2008; Hammond, Linton, Smink, & Drew, 2007; MaclIver & Maclver, 2009). 

A review of the What Works Clearinghouse for practices related to improving graduation 
that included 9" graders yielded seven interventions with positive or potentially positive effects 
(https://ies.ed.gov/ncee/wwc/FW W/Index). Of these, three interventions deliver Tier 2 support 
for specific at-risk subgroups of students, including mentorship programs, policies that place 
students in academic support classes, or behavior support programs such as Check and Connect 
(Christenson, Stout, & Pohl, 2012). Although some of these interventions have demonstrated 
positive impact on student outcomes, they are Tier 2 practices, designed to be implemented with 
a small number of students already identified as at risk (Cauley & Jovanovich, 2006; Neild, 
2009; Sinclair, Christenson, & Thurlow, 2005). The remaining four interventions involve whole 


school restructuring (academies, alternative schools), or are strategies rarely utilized in the first 


year of high school (dual enrollment). In sum, there is a lack of currently identified Tier 1 
interventions for high schools aimed at the prevention of factors that lead to dropout. 

The Institute of Education Sciences (IES) dropout prevention practice guide, however, 
identifies six main practices found to be effective in decreasing high school dropout (Dynarski et 
al., 2008). These practices are in alignment with the MTSS framework and include utilization of 
data systems, instruction and support in areas such as test taking and study skills, personalizing 
the learning environment, and linking classroom content to postsecondary experiences. The 
practices identified by IES in the dropout prevention guide extend the broad framework provided 
by MTSS to include strategies for enhancing student engagement (Dynarski et al., 2008; Goss & 
Andren, 2014), which is critical to dropout prevention (Balfanz, Herzog, & MaclIver, 2007; 
Fredricks et al., 2011). Generally, school engagement can be defined as having three domains: 
behavioral engagement (e.g., doing school work and following rules), cognitive engagement 
(e.g., relevance of school ability to self- regulate, motivation), and emotional engagement (e.g., 
sense of belonging, connections in school; Fredricks et al., 2004, 2011), and any effort designed 
to impact school engagement would address these domains. 


The Freshmen Success Intervention 


The conceptual framework for MTSS emphasizes prevention, data-based decision 
making, the use of evidence based interventions, and implementation fidelity (Sugai & Horner, 
2009). Freshmen Success (FS) was designed by the authors to be embedded within this 
framework to increase student engagement during the 9™ grade, an established critical period for 
early detection and effective intervention. Freshmen Success aims to increase the engagement 
and performance of all 9" graders in high school, regardless of skill level, along trajectories 
toward increased achievement and graduation rates. The core components of FS are: (a) the use 


of data-based decision making by a 9""-grade leadership team, (b) explicit instruction of a 
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prevention-oriented engagement curriculum for all 9"-grade students and (c) utilization of 
engagement-focused peer support from upperclassmen who share experiences and knowledge 
with 9" graders (see Figure 1). 

Data based decision making by a 9‘"-grade leadership team. The establishment of a 9™ 
grade leadership team is at the center of FS, as this team is trained to monitor implementation, 
utilize 9""-grade student outcome data to guide ongoing decision making, and support the 
sustainability of the effort. Effective systems of support use a leadership team to focus on 
articulating successful practices and systems through a review of data, alignment with current 
initiatives, and sharing and gathering feedback from the school staff and leaders (Sugai & 
Horner, 2009). The FS 9'"-grade Leadership Team is made up of a building administrator, three 
to four teachers and staff who work with 9"-graders, and student support personnel such as the 
school psychologist, counselor, or dean of students. The team meets monthly for about 45 
minutes, follows a standard agenda and is responsible for developing consistent 9""-grade-wide 
policies, communicating with the broader school community, and adjusting implementation as 
necessary to fit the school context and ensure sustainability. The primary task of the Leadership 
Team is to utilize data based decision making, a key practice in effective service delivery 
according to the National Association of School Psychologists (National Association of School 
Psychologists, 2010). Key indicators when monitoring for dropout prevention include 
attendance, behavior (e.g., suspension, expulsion), and course performance (e.g., course failure, 
GPA; Allensworth & Easton 2005; 2007; Rosenkranz, de la Torre, Stevens, & Allensworth, 
2014), so the Leadership Team sets and monitors regularly goals for the 9" grade in these key 
areas. Monitoring such early warning indicators has been shown to reduce the percentage of 


students with risk indicators related to chronic absence and course failure (Faria et al., 2017). 


Explicit instruction of engagement skills through the FS Curriculum. The FS 
Curriculum consists of 12 30-minute lessons. As noted earlier, instruction in areas such as test 
taking and study skills, goal setting, problem solving, and decision making is an effective 
strategy identified in the IES dropout prevention practice guide (Dynarski et al., 2008). 
Similarly, a research brief by the Breakthrough Collaborative (2011) states that students “need to 
be explicitly taught the skills and behaviors that will help them navigate the unfamiliar and more 
demanding terrain of high school, including problem solving skills, time management skills, 
organizational skills, self-advocacy, and understanding where and when to seek help” (p. 4). All 
FS curricular instruction is designed to teach these skills and behaviors, with activities that align 
with the three areas of school engagement: behavioral (e.g., prioritization, study strategies), 
cognitive (e.g., on track for graduation, reading transcript) and emotional (e.g., teacher allies, 
getting involved; Fredericks et al., 2004, 2011). 

Peer support delivered by Peer Navigators. The FS intervention includes 
upperclassmen known as Peer Navigators who are trained to support 9"-graders in “show to do 
school” and are placed in each class where the FS curriculum is taught to reinforce curricular 
concepts and support school engagement and learning. As part of typical development, 9""- 
graders are beginning to strive for autonomy, especially from adult influence. As a result, they 
often look to peers for information and support (Daddis, 2011), and their engagement can be 
enhanced through positive relationships with older peers (Dennison, 2000; Karcher, 2005; Wang 
& Eccles, 2012). Peer leaders who are one to two years older have been shown to facilitate 
prosocial and academic development, as these peers understand the school culture and develop 
positive strategies to overcome problems (DuBois, Holloway, Valentine & Cooper, 2002). 


Purpose of the Study 


This article reports the first set of research findings from a randomized controlled trial of 
FS, a Tier 1 intervention model for 9"-graders to increase student engagement, attendance, and 
course performance. Specifically, we hypothesized that the implementation of FS in treatment 
schools would be associated with (a) higher student motivation and engagement, (b) higher rates 
of school attendance, and (c) improvements in identified academic outcomes (credits earned and 
GPA), than for students in control schools who did not receive the intervention. 
To examine the effectiveness of the FS model, we asked the following research 
questions: 
1. What are the effects of the FS model on motivation and engagement? 
2. What are the effects of the FS model on attendance? 
3. What are the effects of the FS model on credits earned? 
4. What are the effects of the FS model on GPA? 
Method 
Settings and Participants 
Four public high schools (two treatment and two waitlist control) located in one state in 
the Pacific Northwest participated in this study during the 2017-18 school year. Schools were 
recruited during the 2016-17 school year, and demographic data were available from that year 
from the National Center for Education Statistics (NCES) database (see Table 1). Treatment 
school | was located in a midsize suburb and treatment school 2 was located in a fringe rural 
area. Waitlist control schools 1 and 2 were located in small cities. Treatment school 2 and 
waitlist control school 2 were located in the same school district. The average student enrollment 
across the treatment schools was 1,518, and the average student enrollment across the waitlist 


control schools was 1,299. None of the schools were eligible for Title I support. There was a 


total of 1,588 9""-grade students across the four schools. Most students were approximately 14 
years old when they entered 9""-grade. Table 2 provides demographic data on the participants. Of 
these students, 854 (401 females and 399 males) were from the two treatment schools. The 
majority of 9" students in the two treatment schools were White (n = 477, 55.9%) or 
Hispanic/Latinx (n = 217, 25.4%). There were 734 9""-grade students in the two wait list control 
schools (347 males and 339 females). The majority were White (n = 460, 62.7%) or 
Hispanic/Latinx (n = 97, 13.2%). 
Measures 

Student motivation and engagement. Student motivation and engagement were 
measured using the Motivation and Engagement Scale - High School (MES-HS; Martin, 2016), a 
student self-report measure of positive and negative factors affecting motivation and 
engagement. MES-HS surveys were administered twice (Time 1: within the first 4 weeks of the 
school year, Time 2: the last 4 weeks of the school year) by classroom teachers, using a paper 
and pencil format. The measure consists of 44 items with a 7-point Likert-type scale, from 1 
(Disagree Strongly) to 7 (Agree Strongly). For example, an item on the MES-HS scale for global 
positive motivation includes “Jf try hard, I believe I can do my schoolwork well” (item 13) and 
an item on the global positive engagement includes “Jf 7 can’t understand my schoolwork at first, 
I keep going over it until I do” (item 1). Items are organized into positive and negative subscales 
(11 subscales; four items per subscale). Several studies have found the subscales of the MES-HS 
to be reliable (Cronbach’s a > .75; see Liem & Martin, 2012, for a review). These subscales can 
then be grouped into four global positive and negative motivation and engagement scale scores 
(average of the subscales scores; Martin, 2016). Based on recommendations by Martin (2016), 


subscales were not created if students answered fewer than three items in a subscale, and global 


scores were not created if students did not have a score for each subscale used to create the 
global scores. A confirmatory factor analysis (CFA) was conducted with a sample of 21,579 high 
school students in 58 Australian schools to validate the psychometric properties of the four 
global MES scales. Findings indicated good model fit (Martin, 2016). Additional evidence of the 
reliability (coefficient alphas) of the four global scales are described below from the study 
sample. 

Global positive engagement. The global positive engagement scale consisted of the 
average of three positive subscales (persistence, task management, and planning). The global 
positive engagement scale also had high reliability for Time 1 (a = .90) and Time 2 (a = .91). 

Global positive motivation. The global positive motivation scale consisted of the average 
of three positive subscales (self-belief, learning focus, and valuing). The global positive 
motivation scale was found to be highly reliable for Time | (a = .91) and Time 2 (a = .92). 

Global negative engagement. The global negative engagement scale consisted of eight 
items and was the average of two negative subscales (disengagement and self-sabotage). Alphas 
for Times 1 and 2 were .86 and .87, respectively. 

Global negative motivation. The global negative motivation scale included three negative 
subscales (uncertain control, failure avoidance, and anxiety). Alphas for the 12 items for Times 1 
and 2 were .86 and .87, respectively. 

Attendance. Attendance was measured as the percent of a school day that each student 
attended in 8th and 9th grade. The mean attendance across each year was used to calculate each 


student’s average attendance in 8" and 9th grades (ranges = 0 — 1.00). 
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Credits earned. Credit data were collected for each full year. Credits earned was 
calculated using the sum of credits earned across semesters 1 and 2 of 9" grade (range = 0 — 
11.50). 

GPA. Student GPA was collected for each semester and included the mean full-year 
GPA across semesters 1 and 2 in 9" grade. All schools were located in the same state and 
measured GPA using a 4-point scale (range = 0 - 4.00) as typical for state reporting standards. 
Table 3 includes sample sizes, means, and standard deviations across these key variables for 
treatment and waitlist control groups. 

Fidelity. Two measures were used to assess the fidelity of implementation of the FS 
model. 

FS Implementation Checklist. Research staff conducted the FS implementation checklist 
monthly with the 9""-grade Leadership team to document fidelity of implementation of the 
components of the FS model. The FS Implementation Checklist was developed by the research 
team with input from a design team, which consisted of administrators, teachers and staff in two 
schools implementing Freshmen Success prior to this RCT. The design team provided ongoing 
input into the usefulness and appropriateness of the items on the FS Implementation Checklist in 
assisting their schools to implement Freshmen Success with fidelity. 

The tool has four subscales: (a) six items that focus on high school systems needed to 
ensure systems-level support and integration of the intervention (e.g., school has an efficient and 
effective system for collecting data and providing reports on critical engagement variables), (b) 
11 items focused on 9”'-grade systems (e.g., 9""-grade staff have received professional 
development in dropout prevention, strategies for utilizing data-based decision making, 


communication systems), (c) five items on the implementation of Peer Navigators (systems for 
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recruitment and training, ongoing support), and (d) four items focused on classroom instruction 
and support (e.g., school explicitly teaches to 9" graders a curriculum focused on increasing 
school engagement within the first semester) related to engagement skills. 

Curriculum Fidelity Observation Form. For each FS lesson, research staff used the 
observation form to document the implementation of each lesson plan component. The form is 
formatted to align with the instructional concepts and activities taught in each lesson, so the 
number of items observed varied for each lesson. Overall, there were five to seven items for each 
lesson rated as taught or not taught and a single rating from 1-10 (1 = poor, 10 = excellent) on 
the overall quality of lesson delivery. 

Procedures 

Recruitment and selection. All procedures were conducted as part of an approved IRB 
protocol at the authors’ institution. The four schools in this study were recruited during the 2016- 
17 school year by research staff. To be eligible, schools needed to be (a) located in the Pacific 
Northwest, (b) have an enrollment over 900 students, (c) have 45% or higher student percentage 
receiving Free or Reduced Priced Meals (FARMS), and (d) have at least 30% minority school 
population, and (e) be within a one hour drive of the authors’ institution. A total of 13 high 
schools met these criteria and were contacted via email and invited to participate. Six schools 
expressed interest, so a researcher-developed capacity assessment was conducted by research 
staff with each of the six schools. This capacity assessment was designed to assess if school 
systems were in place that would be essential to the implementation of FS (e.g., alignment with 
school improvement goals, ability to attain staff consensus on implementation of teaching 
practices, experience with utilizing data for school wide decision making), and if the building 


had identified a period in their schedule for the curriculum to be taught. One school was 
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eliminated after the capacity assessment because it did not meet minimum criteria. Five schools 
went on to complete a researcher-designed systems readiness checklist. The readiness checklist 
was completed by research staff during in-person meetings with school administrative teams. 
The four schools with the highest scores on the capacity and readiness assessments were invited 
to participate. After selection and agreement to participate, four schools were matched into pairs 
based on attendance data in 2016-17 and then randomly assigned to one of two conditions (FS 
model or business as usual) using a coin flip. Two of the schools were located in the same 
district, and the randomization scheme placed each in a different condition. Schools were 
provided funding to offset the cost of training and data collection associated with the study but 
were not otherwise compensated for participation. See Figure 2 for the participant flow diagram. 
Business as usual condition. Business as usual condition was agreed to by the schools 
assigned to this condition through a memorandum of understanding stating that during the 
waitlist period the school would not implement any of the features identified in the memorandum 
(e.g., 9""-grade leadership team, peer support for 9" graders, implementation of an engagement- 
focused curriculum). Adherence to the control condition was documented by the research team 
through a standard interview with each control school administrator following final data 
collection. Interview questions were designed to document any use of the three FS components 
the school may have implemented during the waitlist period. Control schools adhered to the 
memorandum of understanding and did not implement FS components. At the end of the waitlist 
period, neither school had a 9th-grade Leadership Team or similar group that regularly reviewed 
9't_orade data or designed interventions for 9""-graders at the universal level. Neither school had 
a system to utilize upperclassmen to support 9" graders throughout the year, though one school 


included upperclassmen in orientation activities at the beginning of the year. Regarding teaching 
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9" orade expectations, neither school had identified or taught a set of expectations and skills 
needed for success in high school to 9""-graders. 

Implementation of FS model. Each of the treatment schools implemented the three FS 
components. 

FS leadership team. During the spring prior to implementation, teams received 
approximately 8 hours of training from research staff on the use of data-based decision making 
and monitoring and managing system-based interventions. During this early work, teams 
identified specific data-based 9"-grade goals related to attendance, behavior and course 
performance. The team monitored these data goals and the Implementation Checklist during 
monthly 90-minute FS Leadership Team meetings and reported out regularly to staff during the 
implementation year. Either the second or last author attended each team meeting to provide 
technical assistance and general consultation in FS implementation. Each has an extensive 
background implementing MTSS in high schools and providing technical assistance to teams in 
MTSS implementation. Treatment schools averaged 92.5% (range = 90% - 95%) on the 
leadership team subscale of the FS Implementation Checklist in the spring of the year of 
implementation. 

FS curriculum. The FS curriculum was implemented in the fall of the year of 
implementation. Each school had or created a weekly 9""-grade advisory period in the schedule 
for FS lessons to be delivered. These advisory periods took place weekly for approximately 45 
minutes, and all lessons were delivered by 29 advisory teachers during this time. Teachers were 
provided with the curriculum at the end of the prior school year, and then all teachers 
participated in a 4-hour training on the curriculum in the week prior to the start of the 


implementation year. A total of 51 FS lesson observations were completed by trained research 
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staff using the Curriculum Fidelity Observation Form. A sampling procedure was used to ensure 
that all teachers and all lessons were observed at least twice in each school each week until all 
lessons were delivered. The average fidelity of implementation was 94% (range = 40% - 100%; 
SD = 13%). Inter-observer agreement (IOA) was completed on 33% of opportunities and was 
calculated by dividing the exact item agreements by the exact item disagreements plus 
agreements and then multiplying by 100%, with overall IOA being 97%. Both treatment schools 
scored 100% on the curriculum subscale of the FS Implementation Checklist in the spring of the 
year of implementation. 

Peer Navigators. Peer Navigators were recruited by the FS coordinator in each building 
during the spring prior to the year of implementation. Student applications were screened by the 
FS Leadership Team for attendance and grade minimums, and then students were selected, 
ensuring that a diverse range of students would be represented in the overall Peer Navigator pool. 
All students received elective credit for their participation. There were 44 Peer Navigators across 
schools. Peer Navigators received two hours of training prior to the start of the school year on 
their role, the curriculum, and how to work with their partner teacher. Peer Navigators also 
participated in lunch meetings with the FS building coordinator approximately monthly. 
Research staff observed peer navigator implementation during each curriculum observation and 
observed one peer navigator support lunch meeting each quarter. Both treatment schools scored 
100% on the peer support subscale of the FS Implementation Checklist in the spring of the year 
of implementation. 

Analytic Plan 
The study design was a small-scale, cluster randomized controlled effectiveness trial with 


four high schools and randomization occurring at the school level. Because students were nested 
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within schools and preliminary analyses found non-zero ICCs at the school level across all 
dependent variables, we accounted for clustering at the school level in analyses (Peugh, 2010). 
As all dependent variables were continuous, effects of the FS model (control = 0, treatment = 1) 
were evaluated through multi-level linear regressions using the MLR (maximum likelihood 
estimation with robust standard errors) estimator and TYPE=COMPLEX command in Mplus 8.1 
(Muthén & Muthén, 1998-2017). This estimator was selected because it is robust to violations of 
assumptions of linear regression, including non-independence of observations, non-normality of 
observations, and heteroscedasticity (White, 1980; Muthén & Muthén, 1998-2017). 
Multicollinearity was also examined by correlating the treatment group variable (i.e., treatment 
schools vs. waitlist control schools) and the study variables. All correlations were below .80. 

Covariates were included in the regression models when available. For global motivation 
and engagement, global motivation and engagement scores in Time | (beginning of year) were 
included as covariates for scores in Time 2 (end of year). For attendance, 8"-grade attendance 
was included as a covariate for 9""-grade attendance. 

Effect sizes were assessed for the outcome variables regressed on the binary treatment 
condition variable using a web-based meta-analysis calculator (Wilson, n.d.). Using this 
approach, unstandardized regression coefficients of the binary treatment variable, the standard 
deviation of the dependent variable, and treatment group and control group sample sizes were 
imputed, and a Cohen’s d standardized mean difference effect size with confidence intervals was 
calculated (Lipsey & Wilson, 2001). The traditional threshold for a small effect size is .2, a 
medium effect size is .5, and a large effect is .8 (Cohen, 1988), although more recent 


recommendations from the literature note effect sizes of .2 or .25 as educationally meaningful, 
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particualrly for universal interventions (Hedges & Hedberg, 2007; Lipsey et al., 2012; What 
Works Clearinghouse, 2014). 

Missing data. Table 4 includes an overview of the missing data across all outcome and 
covariate variables. Due to the bias introduced through list-wise deletion (Graham, Olchowski, & 
Gilreath, 2007), multiple imputation using Bayesian analyses in Mplus 8.1 was used. Twenty 
data sets were imputed to retain all 1588 cases in the analyses (Muthén & Muthén, 1998-2017). 

Results 

This research examined whether there were significant differences in global positive and 
negative motivation and engagement scores, attendance rates, credits earned, and GPA for 
students in the treatment schools, compared to waitlist control schools. Table 5 includes the 
unstandardized regression coefficients and standard errors for each of the covariates on the 
student outcome variables, as well as effect sizes and confidence intervals. 

Student Motivation and Engagement 

For research question 1, we hypothesized that there would be a significant difference in 
motivation and engagement scores for students in the treatment schools, compared to students in 
the waitlist control schools. Based on the regression analyses, there was a statistically significant 
difference in global positive motivation scores for students in the treatment and waitlist control 
condition conditions (6 = 0.22, p = 002), with a small effect size (d = .21). Treatment conditions 
were also significantly different for global positive engagement (b = 0.17, p < 0.001) and global 
negative engagement (b = -0.30, p < 0.001). Effect sizes were small for both (d= .15 and .24, 
respectively). There were not significantly different negative motivation scores (b = -0.08, p = 
0.073) between treatment and control schools. 


Attendance 
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Our hypothesis for research question 2 was that there would be a significant difference in 
attendance for students in the treatment schools, compared to the waitlist control schools, after 
8""_grade attendance was included as a covariate. As shown in Table 5, results supported our 
hypothesis, as students in the treatment schools had significantly higher rates of attendance than 
students in the control schools (6 = 0.01, p = 0.049). The effect size for treatment condition was 
below the criterion for small (d = .13). 

Credits Earned 

For research question 3, we hypothesized that there would be a significant difference in 
credits earned for students in the treatment and waitlist control schools. Our hypothesis was 
supported (b = 1.13, p = 0.001), as students in the treatment schools earned significantly more 
credits in 9" grade. There was a large effect size for treatment condition on credits earned (d= 
.79). 

GPA 

Our final research question examined whether there was a significant difference in GPA 
across treatment and waitlist control schools. Specially, we hypothesized that students in the 
treatment schools would have significantly higher GPAs. As shown in Table 5, there were no 
significant differences between treatment conditions (b = 1.13, p = .699); thus, this hypothesis 
was not supported. 

Discussion 

High school dropout continues to be a major issue in the US, and students who dropout 

face underemployment, poor health and involvement in the criminal justice system. The path to 


dropout is gradual, making it challenging to identify when to intervene. However, because 
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research indicates that the transition into high school is a particularly vulnerable time, this is also 
an optimal time for preventive intervention. 

Freshmen Success is a Tier 1 prevention-oriented intervention that is implemented within 
a broader MTSS framework and is aimed at increasing student engagement, attendance and 
academic outcomes for 9"*-graders in high school.. A majority of the current interventions to 
improve school completion are intended for students already experiencing failure and in need of 
supports at the Tier 2 or 3 level. The present study extends the research on student engagement 
and dropout prevention for 9" graders by focusing instead on prevention at the Tier 1 level. 
Study results demonstrated a statistically significant difference for the treatment schools on 
student motivation, engagement, attendance, and credits earned compared to schools in the 
control condition. These effects indicate that combining multiple research-supported approaches 
into a cohesive Tier | intervention has the potential to serve as a practice framework for high 
schools. 

Results showed that FS had statistically significant effects on a number of meaningful 
outcomes, with effect sizes considered small by traditional standards but educationally 
meaningful by contemporary standards (Hedges & Hedberg, 2007; Lipsey et al., 2012; What 
Works Clearinghouse, 2014). Such effects are particularly important given that it is a Tier 1 
intervention, where more modest effects are expected because many students already have 
acceptable outcomes (Kraft, 2018). There were statistically significant effects on student 
motivation and engagement, as measured by scores for three of the four global MES variables 
(global positive motivation, global positive engagement, and global negative engagement). More 
specifically, students in the treatment schools showed higher ratings on subscales such as self- 


belief, learning focus, valuing, persistence, task management, and planning and a decrease in 
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rating of subscales related to disengagement and self-sabotage. This finding is encouraging, as 
student engagement predicts student achievement in a variety of settings (Walker, Green, & 
Mansell, 2006). 

Moreover, the finding that implementation of FS resulted in significant, yet small, effects 
on attendance is also encouraging. Poor attendance is a widespread concern. Previous research 
has demonstrated that attendance during 9" grade is one of the most powerful predictors of 
whether a student will complete high school (Allensworth & Easton, 2007). By increasing the 
attendance rates of 9" graders, students benefit from increased instructional time and may 
ultimately experience increased school success and completion. 

Perhaps one of the most exciting findings of the study were the significant effects of the 
intervention on credits earned. Results showed a large effect. “On track” credit accrual in the 
first year of high school is a strong predictor of later graduation outcomes (Allensworth & 
Easton, 2007), and affecting this variable has potential to influence graduation rates. In this way, 
the study on FS adds to the existing literature base and confirms that activities such as team 
monitoring of key student indicators, teaching students how credit accrual works, and 
emphasizing relationship between teacher and students can positively impact credit accrual 
(Neild, 2009; Rosenkranz, de la Torre, Stevens, & Allensworth, 2014). 

Even with significant effects on student engagement, motivation, attendance, and credits 
earned, however, the study did not find implementation of FS to be associated with significant 
improvements in student GPA. The reasons for this finding are unclear. The FS intervention 
monitored student academic progress through a leadership team and taught and reinforced 
academic support skill areas, such as organization and note taking, through a curriculum and peer 


support. It is possible that new academic support skills learned through FS did not sufficiently 
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generalize into the students’ other coursework, or that the intensity of the intervention in this area 
was not strong enough to increase GPA. It is important to explore this further because 9""-grade 
GPA is highly predictive of high school graduation and enrollment in college (Easton, Johnson, 
& Sartain, 2017). 
Limitations 

Although the results show promise for FS, several limitations could be addressed through 
future research. First, the intervention was implemented in a small number of schools. 
Replication of this study with a larger, more diverse sample (e.g., schools in large cities, remote 
rural areas, students with varying socioeconomic backgrounds) would provide more evidence of 
the effectiveness and generalizability of these results. In addition, although there were significant 
effects on student engagement, motivation, attendance, and credits earned, the small intervention 
effect sizes for some of these outcomes make it important to interpret the results with caution. 
Another limitation is there was not a quantitative fidelity measure of business as usual in the 
control schools. A study that included a rigorous fidelity measure in the control schools would 
enhance the methods of this study. 
Implications for Future Research 

The exploration of universal preventive interventions for high schools is still in the early 
stages. The findings from this study attempt to contribute to this emerging research base and 
provide information related to the development of attendance, motivation, engagement and 
course credit accrual in 9""-grade high school students through the FS intervention. Though 
results show promise, a number of research questions could be explored to increase the 
knowledge of the effectiveness and efficiency of this intervention. First, the intervention is 


comprised of three distinct components that were not tested independently of one another. It is 
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possible that the components had varying effects on students, but the research design did not 
allow for such analyses. Future research is needed to examine the impact of individual 
components of the model. Second, the lack of impact on GPA needs further exploration. Follow 
up data collection and analysis are needed to determine if there was deferred impact on GPA in 
later school years, or if additional, more Tier 2 interventions are needed. Third, future research 
could also examine whether the FS model is more effective for different groups of students (e.g., 
students identified as at-risk of school failure, special education identified) and possibly on 
different student outcomes (e.g., course grades, school climate). Last, because the intent of this 
intervention is retention and eventual graduation, longitudinal follow-up studies are needed. 


Implications for Practice 


Many schools, especially at the lower grades, include data teams or MTSS teams in their 
infrastructures to implement a more preventive approach. The current study is one step in 
contributing to a broader understanding of the implementation of Tier 1, prevention-oriented 
supports to support 9" graders. Schools currently implementing a MTSS could embed these 
three FS components (and their current practices and systems) within their current MTSS. For 
example, if a school has a team regularly examining schoolwide data and determining 
interventions at a Tier 1 level, this team could regularly examine 9"-grade data or develop a 
subcommittee do so. Prevention is often considered a key area of focus in primary grades, when 
school psychologists act as instructional experts to prevent inaccurate referrals and establish 
foundational skills for student success. This preventive role is equally important in high schools, 
however, and school psychologists serving high schools can use their data analysis and systems- 
change skills to prevent negative outcomes by informing faculty and staff about the increasing 
data available related to drop out, helping to guide decision making around student outcome data, 


and helping school personnel and families understand the implications if students get “off track.” 
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School psychologists in high schools can attend to the unique needs of 9™ graders in order to 
support interventions like FS designed to address concerns about limited graduation rates. 

There are a number of implications for the practice for schools considering interventions 
to support 9" graders. Freshmen Success combines data based decision making through teaming, 
engagement-focused instruction, and guided peer support. Together, these components provide 
an intervention that aligns with MTSS, the school engagement literature, and the recommended 
practices from IES around dropout prevention. Implementing each with high fidelity can result in 
improved outcomes for 9""-grade students. 

First, school teams can identify relevant data, set benchmarks and monitor the progress of 
their 9'"-grade students as one system-level approach using a problem-solving model. Second, 
these teams, including the school psychologist as a guiding member, can build capacity for their 
school to design and implement solutions that address areas of concern that emerge from the 
data. When implementing FS, schools should begin with the implementation of these freshmen 
leadership teams, as they guide and support implementation of the curriculum and peer navigator 
components. 

When implementing an engagement-focused curriculum such as FS, schools can develop 
lessons that align with the three areas of school engagement and are relevant specifically to their 
students and context. Lesson topics might include available resources in the high school, how 
credit accrual works, and specific study or note-taking strategies that all 9""-grade teachers agree 
to utilize and support in their content area classes. In this study, the curriculum was taught in an 
advisory period, however it could be taught in health class or career or transitions class. Finally, 
implementing the peer navigator component was often the most challenging logistically for 


schools, but also was also very rewarding. Schools can reconsider the training and job 
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descriptions of their upperclassmen Teaching Assistants to act more as student-centered supports 
than clerical teacher supports. In all, combining effective data based decision making by a 9"- 
grade team, teaching engagement-focused curriculum content and providing peer support 
opportunities to all 9" graders as they enter high school and before they fall behind will ensure 


all students are starting with a common level of support and understanding. 
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